TELEPHONE NETWORK-BASED METHOD AND SYSTEM FOR 
AUTOMATIC INSERTION OF ENHANCED PERSONAL ADDRESS BOOK 

CONTACT DATA 


FIELD OF THE INVENTION 

The present invention relates generally to Telephone User Interfaces, and 
specifically to Personal Address Book applications that enable adding contacts for 
usage with enhanced automatic speech recognition via a telephone. 

BACKGROUND OF THE INVENTION 

Telephone answering (TA) applications currently use Automat ic_Speech 
Reco gnition ( ASRJ^ features, which enable a user to add contacts to a Personal 
Address Book (PAB) system via a telephone. One problem is that when a user adds a 
name from the telephone, he cannot assure the accuracy of the spelling of the name, 
because of the inaccuracy of the ASR application feature of the PAB system for 
alphabetic interpretation. For example, the user may receive a voicemail message 
from Joe Phillips, who may be previously unidentified to the user, and who leaves a 
message requesting a callback (the telephone number was received from a PBX, for 
example, and saved in addition to the message itself). The user accesses his PAB, 
and initiates the addition of the new contact based on his speaking both the contact's 
name and the telephone number that was left as part of the message. For example, 
Phillips sounds the same as Philips, Fillips and Filips, and therefore the ASR may 
spell the name incorrectly. 

Even a "smart" system may do no better than "guess" at Phillips, the most 
common member of the homonym group. 

A user is typically a subscriber of a service that is available via the telephone. 
This service typically includes an ASR recognizer and a PAB, and provides for the 
addition of contacts to the PAB directly over telephone. When the user elects to add a 
new contact using the telephone he: 

speaks the contact's name one or more times; and 

enters at least one telephone number to be associated with the name. The 
number can be added verbally by the user directly, or by Dual Tone Multi 
Frequency (DTMF) keypad entry or from the data saved with the message (as 
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received from the PBX) or the number can be added later from the Internet or 

some other device. 
There are two shortcomings for the above process: 

the ASR recognition that results from matching with an alphabetic 
representation of a name can be better than that of contacts added directly from 
the verbal input; and 

by using a directory, e.g. a white pages, the PAB entry can be expanded with a 
growing variety of useful information (e.g. name spelling, address, e-mail, 
other telephone numbers, SMS, etc.). 

The telephone number that was added in one of the ways described earlier may 
not be unique to an individual, since there can be many people sharing this telephone 
number, e.g. a family or a company with one telephone number. 

Where no name is provided, only a telephone number, current TA 
applications can use a directory, such as a white pages directory, based only on the 
telephone number that was entered. If the telephone number is found in the directory 
white pages, the application can insert the spelled version of the associated name into 
the PAB or present it to the user to approve. If many people share a number, there is 
no provision for differentiation between their names, and worse yet, the spelled name 
might not fit the contact. Where the number is for a corporation, for example, the 
spelled name might be that of the company as opposed to being that of the contact 
(the subscriber wants to have the contact's name, although it is a company 
telephone). 

There is, therefore, a need for a PAB application that helps the user verify that 
the added information from an external source (e.g. a white or yellow pages) matches 
his contact. 

SUMMARY OF THE INVENTION 

Accordingly, it is a principal object of the present invention to overcome the 
limitations of existing telephone answering (TA) systems, having Personal Address 
Book (PAB) applications using automatic speech recognition (ASR), and provide 
improved methods and a system for correctly updating these PAB's. 

It is another object of the present invention to help subscribers automatically 


add data to their PAB's, knowing that the automatically added data matches the 
contact's name as well as his telephone number. 

In accordance with a preferred embodiment of the invention, there is provided 
a telephone network-based system for automatic insertion of enhanced contact data in 
a personal address book associated with a user, wherein the user has access to an 
electronic communications network. The system comprising: 

a first application unit to host a telephone network-based system; 
a second application unit programmed to present the user with contact data 
obtained via the electronic network, thereby assisting the user in selecting at least a 
portion of said contact data for entry into the personal address book, the second 
application unit comprising: 

a voice response unit to route a user-initiated call into the system, to 
the second application unit; 

a recognition server comprising an automatic speech recognizer for 
converting to text form a verbalized form of at least one new contact name 
forming part of the user-initiated call; 

a storage unit, which stores at least one textual directory for loading 
into the recognition server; 

a comparison unit for comparing the converted text form with a text 
entry in the at least one textual directory; and 

a textual directory unit for searching the electronic network to locate 
the contact data ba^ed^onjh e ^cony ertedL text formjpf said new contact name , 
such that the selected contact data is inserted in the personal address book 
associated with the user. 

In an exemplary embodiment, a telephone network-based system coordinates 
and optimizes the use of a PAB in conjunction with an ASR, to help identify a new 
contact leaving a voicemail message, and to add this contact as a new PAB entry, 
along with additional useful contact information, such as mailing address, e-mail 
address, alternate telephone numbers and other relevant information. If necessary, 
where a plurality of similar-sounding names exist, the inventive system helps 
determine which of these is correct. 

While adding this information to the PAB entry, the application provides 
assistance to the user during an interactive session involving both a telephone 


interface and a Web interface. In the process of adding the contact information the 
user "trains" the ASR recognizer to associate the verbalized name with the contact 
record. 

The addition of the contact information is achieved by a matching process, 
which uses the ASR capability of the system in the following way: 

• while adding the contact, the user trains the system with his voice, 
and the voice is recorded; 

• uses the telephone number to search external sources for matching 
entries that can be used for augmenting the data; 

• after data is augmented from the external source, e.g. the WEB, 
white pages or a corporate directory, the spelling of the found 
names is fed into the ASR as the words that the ASR should search 
for; 

• the system plays the recorded voice to the recognizer and 
commands it to recognize the name; and 

• if the ASR finds a match between the recorded name and one of the 
names gathered from the external source, a positive match is 
declared. 

The invention assures that the augmented data from any external address book 
(AB) matches the name of the contact spoken by the subscriber. Matching the spoken 
name to the textual name found in the external AB, by using ASR technology, is the 
mechanism by which this assurance is provided. 

The inventive system enables the user to easily add a new PAB entry, and to 
automatically supplement the entry with additional contact data associated with the 
contact, with the knowledge that the added data also "belongs" to the contact. 

For a better understanding of the invention with regard to the embodiments 
thereof, reference is made to the accompanying drawings, in which like numerals 
designate corresponding elements or sections throughout, and in which: 

Fig. 1 is a schematic illustration of a telephone network-based system for new 
contact insertion in a PAB, in conjunction with a Web interface, in accordance with an 
exemplary embodiment of the present invention. 

Fig. 2A is a schematic illustration of the telephone network-based system for 



new contact insertion in a PAB of fig. 1, wherein the APU, RSU and VRU functions 
are distributed over more than one site, in accordance with an exemplary embodiment 
of the present invention. 

Fig. 2B is a schematic illustration of the telephone network-based system for 
5 new contact insertion in a PAB of fig. 1, wherein the APU, RSU and VRU functions 
are distributed over more than one site and for one site they are located within the 
DMZ, in accordance with an exemplary embodiment of the present invention. 

Fig. 3 is a flow chart of the PAB contact addition process for a subscriber 
accessing the system, in accordance with an exemplary embodiment of the present 
10 invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Reference is now made to Fig. 1, which is a schematic illustration of a 
telephone network-based system for new contact insertion in a PAB, in conjunction 
15 with a Web interface, in accordance with an exemplary embodiment of the present 
invention. The system enables the user to operate with the voice-based technology of 
a telephone network, and the text-based technology of the Web, to achieve the 
inventive objects of automatic insertion of contact data into a PAB, in an enhanced 
fashion. 

20 The system 80 involves two main types of interaction with the subscriber, i.e., 

a telephone network interface 55 and an Internet or Web interface 116, which may 
also be in the form of an organizational Intranet interface, such as IBM's global 
virtual machine (VM) network. The following acronyms associated with an 
exemplary embodiment now apply: 


AB 

Address Book 

APU 

Application Unit 

GAB 

Global Address Book 

PAB 

Personal Address Book 

PSU 

Persistence Storage Unit 

RSU 

Recognition Server 

UI 

User Interface 

VoIP 

Voice over Internet Protocol 

VRU 

Voice Response Unit 
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The connection to Internet/Intranet 116 can optionally include a Firewall 114. 
In this case the LAN system is divided into three zones: a LAN secured zone 75, a 
LAN Demilitarized Zone (DMZ) 85 and an Internet Zone 95. 

DMZ is a military term referring to an area between two opponents, where 
fighting is prevented. In a networking context, a DMZ refers to a network added 
between a protected network and an external network in order to provide an additional 
layer of security. However, when a business actually does want someone on the 
Internet to be able to access one of its servers, such as its Web server, an option is to 
place the server in question in a neutral zone between the private network and the 
Internet. Technically, the server is placed on a separate portion (subnet) of the local 
network, and then non-authenticated access is permitted to that specific portion. Such 
a separate portion of the local network is referred to as the De-Militarized Zone, or 
DMZ. 

DMZ 85 becomes an inserted neutral communication zone. Basically, DMZ 
85 is open to most Internet users that can benefit from a company's offerings, its Web 
documents, etc. Firewall (FW) 114 can, however, prevent users from crossing into 
DMZ 85. FW 114 allows no connections to be made directly from Internet zone 95 to 
secured zone 75. The road is simply blocked by the firewall 114. However, Web 
application server 50 located in DMZ 85, when serving a request from Internet 116, 
can access the data and Web application servers 50 in secured zone 75. 

The types of subscriber interfaces are now described: 
i. A telephone Network — this connection is established by a voice response unit 
(VRU) 110, by connecting to a telephone network 55. The connectivity can be via 
European telephony trunks El, or non-European telephony trunks Tl, Voice over 
IP (VoIP) connection type or any other type of telephone connection (such as 
Analog lines etc.). When a subscriber calls into the system, VRU 110 routes the 
call to the user interface (UI) application that is hosted in an application unit 
(APU) 120. APU 120 loads the subscriber data from a persistence storage unit 
(PSU) 130, which, for example, can be a directory server, database or a file system 
comprising a centralized data source. 

At some time before recognition is required, APU 120 commands the 
recognition software that resides in a recognition server (RSU) 125 to load the 


personal address book (PAB) and one or more global address books (GAB's) that 
are relevant to the subscriber. In some implementations APU 120 loads the 
address books (AB) and passes them to RSU 125 rather than RSU 125 loading 
them from PSU 130 directly. 

The ASR software residing in RSU 125 can recognize the names in the 
PAB based on two types of "Contact" enrollment: 

a. The subscriber adds the name using the telephone system via a 
training session. The training session includes the following 
exemplary steps: 

• APU 120 commands RSU 125 to start Voice Enrollment; 

• the subscriber is asked to speak the name of the new 
contact; 

• after prompting the subscriber (i.e., "enter the contact's 
name") VRU 110 passes the PCM (voice) data to RSU 125; 

• the ASR in RSU 125 "learns" the voice pattern and also 
records the original PCM data; it then returns to the 
application a rating number reflecting the quality of the 
training; ^ 

• based on the quality of the training APU 120 decides 
whether to undergo another cycle of training; and 

• when the training is complete, APU 120: 

i. commands RSU 125 to update the PAB data of 

the subscriber with the new contact information 
(so that it can be recognized next time); and 
ii. reads the PCM data from RSU 125 and saves it to 

the PAB to be used to echo the contact's name 
for future verification. 
Note: the actual saving of data to the PAB in i & ii can 
be done either by the ASR or the application. 

b. Add a name, for example, via WEB insertion of the contact name in 
text format as described hereinbelow. In this case the name can be 
saved for recognition in the PAB in one of two ways: 

• use the text representation and load it into RSU 125 when 


required; and 

• partially, or completely, translate the name into a more 
"favorable" format for RSU 125, thereby easing the load 
resource requirement, e.g. less CPU usage and less elapsed 
time. 

At that point, a telephone network-based system session is initiated either 
by the system or by the subscriber who, for example, can then say "Call John 
Jones," where John Jones is an entry in one of the GAB's or the PAB. 

As further described hereinbelow with reference to Fig. 3, the session 
continues in accordance with the steps shown in the flowchart. 

In some alternative implementations the four logical units described above 
can be allocated to different physical units. Any combination is possible, e.g. 
APU 120, VRU 110 and RSU 125 are on one physical unit and PSU 130 on a 
separate physical unit. 

If VoIP is the type of network connectivity being utilized, the VRU can be 
placed in the DMZ zone (as in Fig 2) 

In accordance with the principles of the present invention, once the 
telephone network-based system has added "John Jones" as an entry with or 
without at least one telephone number, the inventive system can access from the 
Web, for example, additional information associated with this contact and 
automatically insert this information into the PAB of the user. 

WEB access, or any other possible way of inserting textual information, for 
example using a corporate intranet - is used in this application for updating the 
PAB or a GAB in PSU 130, with a work telephone number, for example. The 
subscriber, seated at his WEB Client browser 70, which can be a Personal 
Computer (PC), Wireless Access Protocol (WAP) telephone, Personal Digital 
Assistant (PDA) or any other such Web-based personal device, logs into the 
application via the Internet or Intranet 116. Once the subscriber is logged onto the 
Web, an application begins to run in a WEB application server 50. The application 
enables the user to update the PAB or a GAB via the web page so that the 
subscriber can use the updated PAB or GAB via the telephone thereafter. The 
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subscriber usually edits the PAB, and usually someone else that was assigned by 
the organization edits the GAB, wherein the organization members are included in 
the GAB. 

The white pages device 135, shown in figures 1 and 2, is a logical device that 
represents a textual directory unit that can be a source for searching for contact data 
based on a telephone number. This device can, for example, be one of the following, 
or any combination of them: 

a directory server of a database server containing a white pages list or a Yellow 
pages list; 

a Human Resources Database of the organization to which the subscriber belongs; 
a voice mail system list of users; and/or 

a more general search engine function, such as that used by Yahoo that can be 
located in the internet zone with or without some "Proxy agent" in the DMZ zone. 

When a name is added, or edited, via the Web in a text format, it is required to 
save the ASR data in the more "favorable" text format found by means of the Web, 
then the Web application Commands the RSU 125 (or, optionally, another unit 
hosting the ASR engine in the secured or DMZ zone) to "compile" the text name to 
that format and save it in PSU 130 for loading as described earlier (alternatively the 
Web application can ask RSU 125 to compile the data and return the "compiled" 
format, and it is then saved to PSU 130). 

Fig. 2 is the schematic illustration of the telephone network-based system for 
new contact insertion in a PAB of fig. 1, wherein the "front-end" APU, RSU and 
VRU functions are distributed over more than one site, in accordance with an 
exemplary embodiment of the present invention. The system, as shown in Fig 2, can 
thus, alternatively, be spread over multiple locations. For example, PSU 130, DMZ 
85 and white pages device 135 can be in a centralized location, while there are a 
plurality of remote sites 87 for example, Washington and New York, each containing 
a VRU 110, an APU 120 and an RSU 125. All sites 87 have internal LAN's 77, and 
there is a wide area network (WAN) 73 connecting all sites 87. 

When VoIP connectivity is used the VRU can be located in the DMZ zone so 
that access from the Internet zone is enabled. In this case the APU and RSU can be 
located either in the DMZ zone as well or in the secured zone. Site 3, shown in figure 



2B illustrates a case where the VRU, APU and RSU are all located in the DMZ. 

Fig. 3 is a flow chart of the PAB contact addition process, for a subscriber 
accessing the system, in accordance with an exemplary embodiment of the present 
invention. 

5 In block 140, the subscriber adds a contact to the PAB via the telephone. As 

part of the addition of the entry, the system records the subscriber pronouncing the 
name, and the system gets the telephone number of the contact, for example, from the 
voice of the user also pronouncing the number. APU 120, which controls the 
telephony session, searches any external directory that is configured into the system, 

10 and uses the entered telephone number in searching for an entry that matches the 
telephone number. The unit that assists in searching the entry can be for example 
white pages 135, or other relevant address book (AB) directory, such as a yellow 
pages or a corporate directory. In block 142 the system performs a search of the 
contact in white pages 135 using the telephone number of the contact. In block 144 

15 APU 120 determines if the contact is found in white pages 135, and if not, the process 
ends. 

After retrieving all entries that are found to match the telephone number (at 
least one), a capability known in the art, the system 80 operates in accordance with the 
principles of the present invention, to ascertain that the name found matches the name 

20 added in 140. If more than one entry was returned, then the system attempts to find if 
one of the names matches the pronounced name. To do so, APU 120 in block 146 
gets all names from the external AB, and in block 150 loads them to the ASR 
recognizer in RSU 125. 

Subsequently, whether either one or more names are found, in block 160 the 

25 system plays the recorded contact name to the recognizer as was recorded in block 
140. In block 170 the system determines if the played contact name matches one of 
the names found in white pages 135. If no match is found the process ends. If a 
unique match, with quality above a given threshold is found, then in block 180 the 
system fills in all data found in White pages 135, e.g. name, e-mail, address, work 

30 telephone number, etc., into the contact entry in PSU 130. If the process does end, 
then in block 190 APU 120 can decide to train the ASR to recognize this contact 
based on the textual name that was added. The text representation (with or without 
partial compilation) can either be added to the original training, or replace it. In case 


10 


• ft 

the ASR returns a quality that is not above a given threshold, then if this quality 
applies only to one entry, then the user is asked to approve the addition of that entry. 
If there are a few names that are returned with this quality the user is asked to choose 
between them. 

In case the session has ended due to no match found, as in block 170, then 
depending on the system configuration, the subscriber can be offered a name from the 
names found. 

If the telephony session only entered the name of the new contact, but the 
telephone number of the contact was added via a search using Web 116, then the 
previous algorithm is also executed under the control of Web Application server 50. 
In this case RSU 125 can optionally be located in DMZ zone 85, or can even be 
hosted in server 50 itself The answer returned after augmentation is presented on the 
Web page for this contact. If the ASR is not above the threshold quality, a dialog box 
for either approving, or choosing between the possible alternatives, is displayed. If no 
match is found, the list can still be presented, and the user can choose to augment 
from among the listed names. 

It will be appreciated that the preferred embodiments described above are cited 
by way of example, and that the present invention is not limited to what has been 
particularly shown and described hereinabove. Rather, the scope of the present 
invention, as defined by appended claims, includes both combinations and sub- 
combinations of the various features described hereinabove, as well as variations and 
modifications thereof, which would occur to persons skilled in the art upon reading 
the foregoing description, and which are not disclosed in the prior art. 
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